Industry increasingly depends upon highly automated data acquisition and control systems to ensure that industrial processes are run efficiently and reliably while lowering their overall production costs. Data acquisition begins when a number of sensors measure aspects of an industrial process and periodically report their measurements back to a data collection and control system. Such measurements come in a wide variety of forms. By way of example the measurements produced by a sensor/recorder include: a temperature, a pressure, a pH, a mass/volume flow of material, a tallied inventory of packages waiting in a shipping line, or a photograph of a room in a factory. Often sophisticated process management and control software examines the incoming data associated with an industrial process, produces status reports, and, in many cases, responds by sending commands to actuators/controllers that adjust the operation of at least a portion of the industrial process. The data produced by the sensors also allow an operator to perform a number of supervisory tasks including: tailor the process (e.g., specify new set points) in response to varying external conditions (including costs of raw materials), detect an inefficient/non-optimal operating condition and/or impending equipment failure, and take remedial action such as move equipment into and out of service as required. A simple and familiar example of a data acquisition and control system is a thermostat-controlled home heating/air conditioning system. A thermometer measures a current temperature, the measurement is compared with a desired temperature range, and, if necessary, commands are sent to a furnace or cooling unit to achieve a desired temperature.
Typical industrial processes are substantially more complex than the above-described simple thermostat example. In fact, it is not unheard of to have thousands of sensors and control elements (e.g., valve actuators) monitoring/controlling all aspects of a multi-stage process within an industrial plant. These sensors are of varied type and report on varied characteristics of the process. Their outputs are similarly varied in the meaning of their measurements, in the amount of data sent for each measurement, and in the frequency of their measurements. As regards the latter, for accuracy and to enable quick response, some of these sensors/control elements take one or more measurements every second. When multiplied by thousands of sensors/control elements, this results in so much data flowing into the control system that sophisticated data management techniques are required. One currently popular technique is “data streaming.” Here, incoming data are stored, in order of arrival time, in one or more data files. Storing data in time-sequential order allows the control system to quickly access a set of data relevant to the state of a process at a particular time and to make an analysis of this snapshot accordingly.
It is generally desirable to provide as much streaming data as possible to linked (i.e., downstream) data recipients. Once streaming data has been filtered or dropped by an intermediate node, such as an I/O or data acquisition server, it is lost for any subsequent, intended downstream recipients of the data. It is further noted that such data streams are present within a variety of different process control management networks having differing throughput capabilities, connectivity, and reliability. In some networks, the links through which the streaming data passes are fast and always operational. However, in other networks one or more links are relatively slow and/or experience intermittent loss of connections. As a result, providing large quantities of streaming process data to an intended data sink can pose a challenge in the case of certain process management and control network topologies where one or more links in a data stream's path comprise slow and/or intermittent connections. Such challenges can arise in any of a variety of situations including, by way of example: an intermittent connection that has temporarily been suspended, a network interface (e.g., a dialup modem) having a low bandwidth, and multiple application-level processes simultaneously passing data to a single shared network hardware interface for transmission over a network.
For network topologies containing slow/intermittent connections or links, in cases where bursts of data temporarily exceed throughput (including zero throughput for a temporarily down intermittent connection/link), known data acquisition services enter a “store and forward” mode when a data transmission cache reaches a particular level of fullness (measured, for example, by quantity and/or transmission delay). Upon initiation of the store and forward mode, the entire contents of the data transmission cache are moved to the store and forward buffer. Subsequently received data is temporarily re-routed to the store and forward buffer rather than being immediately passed to the data transmission cache for transmission over a network via the slow/intermittent connection.
In such known systems, when the system senses that the network connection is available for forwarding data, data from both the store and forward buffer and the data transmission cache are transmitted over the network connection. In a known system separate threads for the two distinct data sources compete with one another for the potentially limited bandwidth of the network connection—thereby presenting the possibility that the store and forward transmission load will saturate the network connection, re-fill the data transmission cache, and cause the system to reenter the store and forward mode. This known arrangement achieves the desirable effect of preventing loss/dropping of data (assuming the store and forward FIFO buffer is not overrun) when the rate of receiving data for forwarding via a network interface exceeds the current available bandwidth of the network interface resulting in a backup of data to be transmitted via the network interface of a machine. However, entry into the store and forward mode of operation is generally considered an exceptional/undesirable event, and therefore should be avoided.